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Computer Diagnosis and Screening of Psychological and 

Physical Disorders 

Field of the invention 

The present invention relates to a method and system for diagnosis and screening of a 
class of disorders in human subject. These disorders include mood disorders, other 
psychological disorders, drug-induced disorders and physical disorders. 

Background of the invention 

Expressions and movements of the face and other parts of the body (ie. "body language") 
can often demonstrate the psychological and physical state of a human patient. Facial 
expressions, for example, can be used to express emotion, pain, happiness, dissatisfaction and 
many other forms of communication between humans. Additionally, facial expressions may 
also indicate the emotional state of a human subject in addition to other physiological states 
such as brain function. Additional clues to psychological and physical state are provided by the 
patient's choice of words and mode of speaking. 

A large number of individuals in society suffer j&rom various forms of mood disorder. 
For example, severe melancholic depression often results in the partial incapacitation of 
individuals. The treatment of mood disorders is a significant burden on the health system which 
often requires extensive interactions with patients with the associated significant cost. Further, 
there is often a high level of subjectivity in any assessment system which may mean that it is 
difiBcult to measure progress of patients over an extended period of time. The subjectivity of 
any assessment tends to interfere with its accuracy especially where the assessor is replaced 
over time and a subsequent assistance provider must rely on notes of assessments etc. 
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Body language and verbal cues are also vital in the diagnosis of other psychological 
disorders, drug-induced disorders, and physical disorders (eg. stroke, Parkinson's disease). 

Summary of the invention 

It is an object of the invention to provide a system and method which efficiently assesses 
a class of disorders in a human subject. These disorders include mood disorders, other 
psychological disorders, drug-induced disorders and physical disorders. 

In accordance with a first aspect of the present invention there is provided a method of 
determining a disorder measure in a patient comprising the steps of (a) recording temporal data 
associated with the patient over a predetermined interval; (b) computer processing the temporal 
data to determine a series of indicator measures associated with the^data; and (c) comparing the 
indicator measures with those of other patients so as to determine a the mood disorder measure 
associated with the patient. 

The temporal data ideally includes a visual and audio interview with the patient. 

According to a further aspect of the present invention, there is provided a method, to 
identify a disorder in a human subject comprising the steps of: 

recording a series of visual images of a selected body part of a human subject 
sequentially taken over a predetermined time period; 

analysing the series of recorded visual images to determine the degree of change to the 
selected body part over the predetermined time period; 

comparing the image changes of the selected body part with pre-recorded data to 
determine whether or not the human subject suffers from a mood disorder. 

Optionally the method of the invention may further include the step of using the first 
visual image of the series of visual images to correlate the position of the body part of the 
human subject with the body part displayed in the other visual images of the series. This allows 
the changes in the movement of the body part to be tracked throughout the series of images 
taken over a given time period. 
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In one embodiment of the invention, the position of an array of tracking points of the 
body part are recorded in the first image of the series and the displacement of the array of 
tracking points from the first image in each subsequent visual image of the series, may be 
recorded. The degree of displacement of the body part can be recorded as data and compared 
with the data from a human subject who does not suffer from a mood or emotional disorder 
(such as for example, melancholic depression), to thereby identify that whether the human 
subject suffers from the mood or emotional disorder. 

The selected body part may be the facial area of a human subject and may include the 
central facial features of the eyes, mouth and nose. 

In another embodiment of the invention, in order to record the changes of the body part 
of the human subject over a given time period, a tracking area may be selected for the first 
image and the brightness of the tracking area can then be recorded for that image and each 
corresponding tracking area in each subsequent visual image of the series. The series of visual 
images may be pixel images represented in a computer graphic display and the brightness can 
then be determined by counting the number of non-dark pixels in the tracking area compared to 
that of dark pixels. 

According to a second aspect of the present mvention, there is provided a system to 
identify a mood disorder in a human subject comprising: 

a visual data recording means to record a series of visual data images of a selected body 
part of a human subject sequentially taken over a predetermined time period; 

a data processing means capable of receiving the visual data images from the visual data 
recording means, the data processing means capable of analysing the series of recorded visual 
data images to determine the degree of change to the selected body part over the predetennined 
time period; and 

a comparative data means in data communication with the data processing means, the 
comparative data means capable of comparing the image changes of the selected body part with 
pre-recorded data to determine whether or not the human subject suffers from a mood disorder. 
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Where in the description of the specification the word "comprising" or "comprised" is 
used, unless otherwise stated explicitly, the word is to be interpreted inclusively rather than 
exclusively. 

The preferred embodiment can be adapted to be a diagnostic aid in the treatment of other 
psychological disorders, drug-induced disorders, and physical disorders such as stroke and 
Parkinson's disease. 

Brief description of the drawings 

Notwithstanding any other forms which may fall within the scope of the present 
invention, preferred forms of the invention will now be described, by way of example only, with 
reference to the accompanying drawings in which: 

Fig. 1 illustrates a flow chart of the steps of the preferred embodiment; 

Fig. 2 is a graphical plot of data showing the facial displacement of a normal patient 
compared to a patient with melancholic depression, over a series of visual frame images; 

Fig. 3 is a graphical plot of data showing the facial brightness changes of a normal 
patient compared to a patient with melancholic depression, over a series of visual jQrame images; 
and 

Fig. 4 is a graphical plot of data showing a grid deformation measure of a normal patient 
compared to a patient with melancholic depression, over a series of visual frame images. 

Detailed description of the embodiments 

An embodiment of the invention provides a method and system for identify melancholic 
depression in a human subject. A series of visual images of a facial area of a patient are firstly 
recorded for a predetemained time period and the data stored in a computer so that a software 
application program can analyse visual images of the facial area, to thereby determine the 
changes which take place to the facial area over the predetermined time period. A comparison 
is then made with the facial changes of the facial area of the patient, with that of pre-recorded 
data to determine whether or not the patient could possibly suffers from a mood disorder. 
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Therefore the steps involved in the preferred embodiment are illustrated in Fig. 1 with a 
first step being the capture of a series of video images of a patient 1, followed by a temporal 
processing of the facial portions of the images 2 to derive various measures, followed by the 
outputting of indicative values 3. 

The embodiment of the invention has been applied to the mood disorder of depression 
and in particular to a sub-type of depression known as melancholic depression. The 
embodiment of the invention provides an assessment of psychomotor changes associated with 
melancholic depression in a human patient. That there are several physical changes associated 
with melancholic depression which includes slow movement, postural slumping and relative 
immobility of the face and body. These physical changes are used to identify melancholic 
depression utilising computer automated methods. 

The method was developed and tested on six sequences of video taken from human 
patients. The video sequence was captured at a rate of twelve frames per second with a duration 
up to ten seconds for each sequence. Each sequence consisted of images of a seated subject 
with pose varying from looking straight at the camera to looking at approximately 45 degrees 
from the direction of the camera. 

Each of the pictures for each sequence were recorded as data images by a digital camera 
and the data images input to a file stored on the hard drive of a computer (not shown). 



After the data has been stored in the computer, an assessment was made of the series of 
data images for each sequence so as to determine the degree of the facial mobility and changes 
of the patient. This involves tracking the face location through the video sequence. The data 
images therefore represent a series of image frames which represent the visual images of the 
patient's face. 

One method of face tracking can be as follows: 

Initially, the tracking in the first frame was achieved by placing a virtual box around the 
central facial features constituting the eyes and mouth to thereby define a tracking area of the 



Tracking 
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patient. Correlation techniques are then used to track the virtual box location through the 
subsequent frames by keeping the box size of the sequence constant for each data image in the 
sequence. 

In a first automated measurement facial motion was measured from frame to frame of 
each sequence by measuring the frame displacement of the tracked face region after correlation 
processing. Fig. 2 illustrates the resultant displacement of the facial region for each frame 
which is plotted graphically. The changes are computed with a two frame lag, that is a frame 
rate of three frames per second. 

The plot of Fig. 2 shows the frame number of a frame sequence plotted against the facial 
displacement of facial features of two sequences, the solid line representing a normal facial 
displacement (the data being stored in a data base on the computer) and the other plot showing 
the case for severe facial immobility. The facial motion measure as can be seen in Fig. 2, is 
significantly higher for the normal sequence than for the immobile sequence of the patient 
suffering from melancholic depression. Therefore this measure represents making a comparison 
of these two groups of data it is possible to infer whether a patient is likely to be suffering from 
melancholic depressions. 

Another way to identify facial changes in a sequence of visual frames is to measure the 
"monitor brightness changes" within the tracked face region on a computer display. One 
method to monitor the brightness changes within the virtual box is to sum the magnitude of the 
pixel differences over the area defined by the virtual box. To reduce the noise levels which may 
be inherent in this method, an altemative brightness monitoring method was developed. This 
method proceeded by differencing each facial image from a previous image so as to produce a 
different map image, calculating the standard deviation of the difference map image and 
counting the number of difference map pixels that are more than 4 standard deviations from 
zero. The count is then normalised by dividing the area of the face virtual box. 

Fig. 3 illustrated a plot of the frame to frame brightness changes using this method for 
the same sequences used in Fig. 2. The changes are computed with a two frame lag. It can be 
seen that the sequence of the normal subject shows significantly larger facial brightness changes 
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then the sequence of the severaUy impaired subject which suffers from melanchoUc depression 
and consequentiy it is possible to identify that the patient suffers from melancholic depression. 

A third method for measuring facial activity involves using a series of 5x5 grid points 
placed on the first of a pair of visual images which are then tracked by correlation to the second 
visual image within the sequence of images. Constraints are placed on the grid pomts to ensure 
that the basic grid structure is retained in the tracked image and points don't "cross-over" other 
grid lines. 

Fig. 4 shows a plot of the amount of grid deformation against frame number using the 
same data as used in the groups of Figs. 2 and 3. Again there is a clear distinction between the 
melancholic depression sufferer and the non-sufferer. 

A Composite Measure 

In order to get a simple composite summary of the performance of the proposed 
techniques on a series of test sequences, the mean value of each measure for a given sequence 
was calculated to produce a sequence score by taking the mean of the three methods. The 
scores for each sequence are given in Table 1 below. 



TABLE 1 


Sequence 


CORE rating (Facial 
Immobility) 


Sequence Score 


1 


0 


2.8 


2 


3 


1.2 


3 


2 


1.5 


4 


2 


1.3 


5 


0 


2.5 


6 


0 


2.7 
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As can be seen from Table 1, the summary scores for each video sequence shows a 
strong inverse relationship to the core assessments for facial immobility. 

As will be appreciated the present embodiment provides a means of identifying a mood 
disorder in a human subject This embodiment could be implemented for measuring other body 
parts of a human subject other than the facial region of the tracking box which mcludes the 
eyes, nose and mouth portion of a human subject. For example other tracking areas may 
include body motion, hand motion, posture, gate, gaze. Furthermore, in odier embodiments of 
the invention, the speaking patterns of the human subjects may be combined with the visual 
data to determine mood disorders. 

This embodiment provides a visual method and system for identifying mood disorders 
and may be implemented in clinical rooms, intemet telepsychiatry, research environments and 
in hospitals. 

It would be appreciated by a person skilled in the art that numerous variations and/or 
modifications may be made to the present invention as shown in the specific embodiments 
without departing from the spirit or scope of the invention as broadly described. The present 
embodiments are therefore, to be considered in aU respects to be illustrative and not restrictive. 

It would also be appreciated by a person skUled in the art that the technology described 
in this invention could be adapted to be a diagnostic aid in the tireattnent of other psychological 
disorders, drug-induced disorders, and physical disorders such as sti-oke and Parkinson's disease. 
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Claims 

1. A method of determining a disorder measure in a patient comprising the steps of: 

(a) recording temporal data associated with said patient over a predetermined interval; 

(b) computer processing the temporal data to determine a series of indicator measures 
associated with the data; and 

(c) comparing said indicator measures with those of other patients so as to determine a 
said disorder measure associated with said patient. 

2. A method as claimed in claim 1 wherein said temporal data includes a visual 
interview with said patient 

3. A method as claimed in any previous claim wherein said temporal data includes 
audio responses of said patent to a series of questions. 

4. A method to identify a disorder measure in a human subject comprising the steps 

of: 

recording a series of visual images of a selected body part of a human subject 
sequentially taken over a predetermined time period; 

analysing the series of recorded visual images to determine the degree of change 
to the selected body part over the predetermined tiine period; 

comparing the image changes of the selected body part with pie-recorded data to 
determine a mood disorder measure for said patient. 

5. A method according claim 4, wherein the method further includes the step of 
using the first visual image of the series of visual images to correlate the position of the body 
part of the human subject with the body part displayed in the other visual images of the series. 

6. A method according to claim 4 or 5, wherein the position of an array of tracking 
points of the body part are recorded in the first image of the series, and the displacement of the 
array of tracking pomts firom the first image in each subsequent visual image of the series, is 
recorded. 
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7. A method according to claim 4 or claim 5, wherein the selected body part is the 
facial area of a human subject. 

8. A method according to claim 4 or claim 5, wherein the facial area includes the 
central facial features of the eyes» mouth and nose. 

9. A method according to any one of claims 6 to 7, wherein a tracking area is 
selected for the first image and the brightness of the tracking area is recorded for that image and 
each corresponding tracking area in each subsequent visual image of the series. 

10. A method according to claim 9, wherein the series of visual images are pixel 
images and the brightness is determined by counting the number of non-dark pixels in the 
tracking area compared to that of dark pixels 

11. A method as claimed in any previous claim wherein said disorder comprises one 
of a mood disorder, psychological disorder, drug-induced disorder or physical disorder. 

12. A system to identify a disorder in a human subject comprising: 

a visual data recording means to record a series of visual data images of a selected body 
part of a human subject sequentially taken over a predetermined time period; 

a data processing means capable of receiving the visual data images from the visual data 
recording means, the data processing means capable of analysing the series of recorded visual 
data images to determine the degree of change to the selected body part over the predetermined 
time period; and 

a comparative data means in data communication with the data processing means, the 
comparative data means capable of comparing the image changes of the selected body part with 
pre-recorded data to determine whether or not the human subject suffers from a mood disorder. 
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